Our client, a multinational food and drink processing conglomerate, partnered with us to optimize their trade promotion management (TPM) and trade promotion optimization (TPO) using Azure Databricks. HCLTech enabled the client to perform effective trade promotion execution (TPx) by delivering Integrated Data Management services, integrating data from various sources and managing their data infrastructure. As a result, our client achieved 70%-80% time and cost efficiency, 20%-30% improved decision-making and 10x faster processing.
The Challenge
High operational costs and inefficient processes
Our client faced challenges with their existing on-premises SAP CRM system used for trade management. The marketing and sales data were scattered across various applications, creating silos and hindering seamless data sharing with third-party service providers. This fragmentation led to high operational costs, reduced efficiency, slower processes and diminished accuracy. This further induced counter-intuitive data management practices within the organization, as teams owned, managed and organized their data differently without reusing the information across departments. This lack of standardization led to cost optimization issues, as some teams faced high storage and low compute costs, while others encountered low storage and high compute costs.
The Objective
Accelerate sales growth and implement effective marketing strategies
Our client sought to modernize their end-to-end technology infrastructure while managing high volumes of data for emerging markets by transitioning to a unified, scalable cloud-based analytics platform. We aimed to provide trade promotion services through targeted marketing campaigns that would influence purchasing decisions across all sales channels. Key components of these campaigns included data availability, data sharing, data quality and data integration to drive impulse buying, promote product value, create product bundles and offer retailer incentives.
The ultimate goal was to accelerate sales growth and implement effective marketing strategies to expand global outreach. We leveraged Databricks on Azure to enable dynamic and accurate data processing, optimizing ROI, reducing costs and enhancing overall efficiency.
The Solution
Unifying the data ecosystem with Databricks Lakehouse platform
We proposed Databricks as it provided a flexible, modern, next-gen cloud-based analytics platform capable of integrating and analyzing structured and semi-structured data for optimized campaign planning, improved extensibility and scalability. Azure Databricks facilitated various forms of analytics, such as BI-descriptive, AI and ML-Predictive, that could not be executed in our client's existing SAP model. Our client sought a standardized, collaborative, single-environment platform with a single source of truth for data scientists, analysts, engineers, marketing and sales teams, all supported by a consistent data set.
A key feature of the implemented solution was its configuration-based design, allowing it to be easily implemented across different markets without significant changes to existing code. We optimized data storage by using a compact format to reduce size, maintain high availability and lower costs. The solution also supported data across all markets without the need for manual synchronization.
Our team built a parameterized data pipeline capable of fetching diverse data, such as customer, material and pricing information, for all markets. We utilized Delta Lake for data storage and analytics capabilities in both existing and new markets. Additionally, we employed Databricks to identify and generate accurate delta records, addressing our client's lack of tools and methods for this task.
We achieved an end-to-end integration with enterprise systems, which improved data visibility across teams through standardization and automation. This allowed resources to focus on more critical tasks and perform continuous monitoring of data pipelines. As a result, the problem of distributed costs was successfully resolved.
The Impact
Faster processing, cost efficiency and improved decision making
The implementation of Databricks and Delta Lake had a significant impact on our client's operations, leading to substantial improvements in productivity and efficiency:
- By leveraging the cloud-based architecture, they were able to process billions of records in just a few hours using Databricks' data lake architecture, compared to the days or weeks previously spent on pipeline jobs with SAP
- Automating workflows and building generic pipelines using Databricks contributed to a 70%-80% gain in time and cost efficiencies, eliminating the need for code changes
- Decision making in trade optimization was also enhanced, with a 20%-30% improvement achieved through automation using Databricks
- The decommissioning of the Azure SQL Server and transition to Databricks Delta Lake resulted in a 20%-30% decrease in costs
- Moreover, Delta Lake implementation on Databricks facilitated a 10x increase in decision making and performance
